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The  segmentation  algorithm  proposed  in  this  paper 
is  a complex  form  of  thresholding  which  utilizes 
multiple  thresholds.  The  algorithm  consists  of  two 
major  components:  a threshold  selection  component  and 
a relaxation  component. 

The  threshold  selection  component  automatical ly 
selects  a threshold  so  as  to  maximize  the  global 
average  contrast  of  edges  detected  by  the  threshold 
across  the  image  This  algorithm  for  threshold 
selection  compares  favorably  with  other  methods  for 
automatic  threshold  selection.  The  threshold  selection 
algorithm  can  be  applied  recursively  to  select 
additional  thresholds  by  ignoring  any  edges  which  have 
already  been  detected  by  previously  selected 
thresholds. 

The  relaxation  component  utilizes  the  immediate 
spatial  context  of  each  pixel  to  update  both  the  label 
at  the  pixel  and  the  feature  measurement  at  the  pixel. 

The  update  function  proposes  a new  feature  value  at  the 
pixel  defined  by  a weighted  average  of  the  central 
pixel  and  all  of  its  neighbors.  The  weight  associated 
with  each  pixel  <with  respect  to  the  pixel  being 
updated)  is  proportional  to  the  spatial  distance 
between  the  pixels>  the  probability  that  the  two  pixels 
are  correctly  labeled,  and  the  probability  that  the  two 
pixels  belong  to  the  same  region.  The  update  function 
then  replaces  the  feature  value  at  the  pixel  with  a 
value  somewhere  between  the  current  value  and  the 
proposed  value.  When  the  local  evidence  for  shifting 
the  feature  value  is  consistent  then  the  value  adopted 
will  be  close  to  the  proposed  value;  however,  when  the 
local  evidence  is  inconsistent  the  value  adopted  will 
be  close  to  the  original  value. 

The  relaxation  is  independently  performed  for  each 
threshold  ^elected.  The  resulting  binary  images  are 
intersected  to  produce  the  final  segmentation.  This 
algorithm  works  well  not  only  for  simple  images  but 
also  produces  reasonable  segmentations  for  complex 
images. 
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A SEGMENTATION  ALGORITHM  BASED  ON  THRESHOLDING 


1 0 Introduction 

This  paper  develops  a segmentation  algorithm  based 
on  multiple  thresholds  An  image  segmentation 
algorithm  partitions  an  image  into  disjoint  sets  of 
spatially  contiguous  pixels  (referred  to  as  regions). 
The  goal  of  image  segmentation  algorithms  is  to  produce 
segmentations  for  which  there  is  a high  corellation 
between  the  entities  of  the  real  world  (objects/ 
surfaces,  and  parts  of  objects)  and  the  regions  of  the 
segmentation  Existing  image  segmentation  algorithms 
can  be  divided  into  two  broad  classes.  The  first  class 
attempts  to  build  regions  in  the  image  based  on  the 
similarities  of  some  characteristics  (or  features)  of 
the  picture  elements  (pixels)  in  the  image  The  second 
class  of  algorithms  attempts  to  locate  those  edges  in 
the  image  which  correspond  to  object  or  surface 
discontinuities  Cfn  11  based  on  differences  between 
pixel  characteristics 

The  segmentation  algorithm  described  here  does  not 
strictly  lie  in  either  of  these  two  classes.  The 
algorithm  attempts  to  utilize  both  pixel  similarity  and 
pixel  difference  information  to  arrive  at  a meaningful 


[13  Henceforth#  both  object  and  surface  boundaries 
will  be  referred  to  as  object  boundaries.  While 
there  are  particular  considerations  relevant  to 
the  detection  of  surface  boundaries,  they  are  not 
the  focus  of  this  treatment. 
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segmentation  It  is  hoped  that  the  quality  of 
segmentations  based  on  both  tynes  of  knowledge  will  be 
superior  to  segmentaions  based  only  on  pixel 
similarities  or  pixel  differences 

The  basic  mechanism  used  to  generate  the 
segmentation  is  a complex  form  of  thresholding.  Let  us 
consider  some  of  the  issues  in  the  simple  thresholding 
of  images  The  image  is  partitioned  by  assigning  one 
label  to  pixels  with  feature  values  which  are  above 
some  threshold  T.  and  another  label  for  pixels  with 
feature  values  not  above  T.  For  some  images.  such  as 
chromosome  images  or  hand  printed  characters,  where  a 
clear  foreground-background  (figure-ground) 
relationship  exists,  a single  threshold  will  be  able  to 
detect  all  or  most  of  the  object  boundaries  at  the 
object-background  discontinuity.  However,  those 
boundaries  which  correspond  to  object-object 
discontinuities.  or  internal  structure  of  the  object 
may  not  be  detected  by  that  threshold  Furthermore,  in 
more  complex  images  which  do  not  exhibit  a clear 
foreground-background  distinction  (such  as  images  of 
natural  outdoor  scenes),  one  cannot  expect  a single 
threshold  to  detect  all  or  even  most  of  the  object 
boundaries  in  the  scene  In  order  to  detect  most  of 
the  interesting  boundaries  within  an  image,  our 
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segmentation  algorithm  will  be  defined  in  terms  of  a 
set  of  n thresholds  rather  than  a single  threshold 
These  n thresholds  will  partition  the  feature  space 
into  n+1  possible  classes 

Section  20  below  classifies  the  possible 
segmentation  errors  assuming  a 'correct'  segmentation 
is  known  These  error  classes  are  an 

over s imp  1 1 f i ca 1 1 on  since  there  are  no  accepted  criteria 
defining  a 'correct'  segmentation)  however,  they  aid 
in  the  analysis  of  the  segmention  processes 

Figure  1 shows  a block  diagram  of  the  structure  of 
the  proposed  segmentation  algorithm  The  algorithm 
selects  a set  of  n thresholds,  independently  applies  a 
relaxation  correction  procedure  to  each  binary 
segmentation  defined  by  the  set  of  thresholds,  and 
combines  the  resulting  n segmentations  into  a single 
segmentation  The  threshold  selection  component 
au toma t i c a 1 1 y selects  a set  of  thresholds  so  as  to 
maximize  the  global  average  contrast  of  edges  detected 
by  the  threshold  across  the  image.  The  development  of 
the  algorithm  and  a comparison  to  other  threshold 
selection  algorithms  is  presented  in  sections  3.0  to 
3.  5 below  The  thresholder  in  figure  1 simply 
generates  a binary  labeling  of  the  image  pixels  for 
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each  selected  threshold  The  relaxation  process 
utilizes  the  immediate  spatial  context  of  each  pixel  to 
update  both  the  label  and  the  feature  measurement  at 
the  pixel  The  relaxation  process  can  be  viewed  as  an 
interpolation  between  the  central  pixel  and  the  set  of 
pixels  in  the  spatial  context  of  the  central  pixel 
The  development  of  the  relaxation  process  is  described 
in  detail  in  sections  4.0  to  4.7  below  To  obtain  the 
final  segmentation,  the  intersector  process  combines 
the  set  of  binary  segmentations  by  simply  overlaying 
them  and  defining  a new  region  label  for  each  distinct 
combination  of  n labels  in  the  set  of  binary  images 

2.0  Segmentation  Errors 

Given  any  resultant  segmentation  and  a 
corresponding  'correct'  segmentation  tfn  21  of  an 
image,  one  can  distinguish  two  primary  types  of  errors 
The  segmentation  can  contain  boundaries  which  do  not 
exist  in  the  'correct'  goal  segmentation  and. 
therefore.  do  not  correspond  to  any  real  object 
discontinuity  in  the  image  (i  e a false  alarm). 
Alternatively.  the  segmentation  can  miss  edges  which 

[21  Note  that  there  are  no  well  defined  criteria  for 
'correct'  segmentation  of  an  image  Furthermore, 
the  definition  of  a 'correct'  segmentation  is 
inherently  ambiguous  since  the  'correct' 
segmentation  is  a function  of  the  goals  of  the 
segmentation  system. 


A SEGMENTATION  ALGORITHM  BASED  ON  THRESHOLDING  PAGE  5 


appear  in  the  'correct'  segmentation.  These  will  be 
referred  to  as  tune  gng  and  tuac  two  errors 
respectively  Examples  of  these  errors  are  provided  in 
figure  2 This  figure  also  shows  a third  type  of  error 

which  can  be  viewed  as  a compound  type  one  and  type  two 

error  For  this  type  of  error  the  boundary  in  the 
'correct'  segmentation  is  correctly  detected  by  the 

segmentation  algorithm,  but  not  in  the  proper  location 
These  errors  are  often  due  to  inaccuracies  in  the 

transf ormat 1 on  between  the  spatially  continuous  image 
and  its  discrete  representation  It  should  be  much 
easier  to  recover  from  these  errors  than  from  type  one 
or  type  two  errors 


1 


20  Threshold  Selection 

The  threshold  selection  process  should  select  that 
threshold  which  minimizes  some  measure  of  the  expected 
segmentation  errors  The  proposed  threshold  selection 
algorithm  is  based  on  estimating  the  expected  number  of 
type  one  and  type  two  errors  using  either  edge 
information  or  pixel  feature  differences  This 
algorithm  will  be  compared  to  several  other  algorithms, 
most  of  which  attempt  to  minimize  the  number  of  pixels 
mi  sc  lass i f i ed  using  pixel  similarity  information 
Since  the  edge  and  region  approaches  are  based  on 
totally  different  models  of  image  information.  it  is 


Type  three  error  - Incorrectly  placed  edges 
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drgued  that  the  range  of  images  to  which  automatic 
threshold  selection  is  applicable  is  extended 

Note  that  the  problem  of  error  estimation  is 
considerably  more  complicated  when  multiple  thresholds 
and  processing  after  the  thresholding  are  considered 
Estimating  errors  for  a given  threshold  becomes 
difficult  since  apparent  type  two  errors  may  be 
detected  by  secondary  thresholds  Furthermore,  some 
errors  (especially  type  three  errors)  may  be  corrected 
by  the  post-thresholding  relaxation  process 


3.  1 Threshold  Selection  Algorithms 

This  section  introduces  several  methods  for 
automatic  threshold  selection  for  comparison  to  the 
methods  proposed  in  section  32  below  Both  the 
comparison  threshold  selection  methods  and  the  proposed 
threshold  selection  methods  are  summarized  in  table  1 

One  standard  method  for  threshold  selection  (Ml) 
utilizes  a histogram  of  the  values  of  the  selected 
features  The  algorithm  assumes  that  the  regions  to  be 
detected  by  the  threshold  differ  in  terms  of  their 
feature  activity,  and  that  therefore.  different  peaks 
in  the  feature  histogram  correspond  to  different  image 
regions  This  assumption  is  often  violated,  especially 


Table  1 Summary  of  Threshold  Selection  Algorithms. 


Ml  Standard  Feature  Histogram  Method 

This  method  selects  a valley  between  two  peaks  in  the 
feature  histogram 


M2.  Gradient  Weighted  Feature  Histogram  Method 

This  method  is  identical  to  method  Ml  except  that  the 
contribution  of  the  pixel  to  the  feature  histogram  is 
weighted  toward  pixels  of  low  gradient. 


M3  High  Gradient  Pixel  Histogram  Method. 

This  method  is  identical  to  method  Ml  except  that  only 
image  pixels  exhibiting  high  gradient  are  considered  in 
the  hi stogram. 

M4.  Total  Gradient  Histogram  Method. 

This  method  selects  the  threshold  at  the  largest  peak  in 
a t o ta 1-grad i ent  histogram.  The  histogram  is  computed 
as  for  method  Ml  except  that  the  contribution  of  a pixel 
to  the  feature  histogram  is  directly  proportional  to  the 
gradient  magnitude  at  the  pixel. 

M5.  High  Gradient  Pixel  Average  Method 

This  Method  computes  the  threshold  by  simply  averaging 
the  feature  values  of  all  high  gradient  pixels. 

M6.  Uniform  Average  Contrast  Method. 

This  method  selects  the  threshold  which  generates  the 
highest  average  contrast  of  detected  edges  over  the 
image  using  the  expected  contrast  histogram. 

M7.  Relative  Contrast  Method. 

This  method  is  identical  to  method  M6  except  that  the 
contrast  of  detected  edges  used  in  computing  the 
histogram  is  defined  relative  to  the  threshold  rather 
than  using  the  simple  edge  strength  as  the  measure  of 
contrast. 
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when  high  levels  of  texture  are  present  in  the  image. 
Nevertheless,  let  us  consider  the  case  for  which  the 
assumption  is  valid  and  the  peaks  in  the  feature 
histogram  correspond  to  overlapping  normal 

distributions  of  feature  activity.  With  two  normal 
d i s tr i b ut i ons  the  Bayesian  minimum  error  decision  would 
place  the  decision  boundary  (i.e.  the  threshold)  at 
the  minimum  between  the  histogram  peaks.  The  correct 
application  of  such  statistical  methods  is  virtually 
impossible  in  practice  due  to  the  difficulty  in 
estimating  the  underlying  distributions  from  the 
histogram;  especially  when  the  number  of  underlying 
distributions  is  unknown  or  when  the  types  of  the 
underlying  distributions  are  unknown.  Therefore,  the 
simple  heuristic  of  threshold  placement  at  the  minimum 
between  histogram  peaks  is  typically  adopted  Lfn.  31. 
This  method  of  threshold  selection  is  referred  to  as 
the  standar d h i stoaram  method.  There  are  two 

difficulties  in  threshold  selection  using  this  method. 

a)  the  valleys  between  histogram  peaks  are  long 
and  flat  making  threshold  selection  difficult; 
and  b)  the  edge  information  in  the  image  is  not 
uti  1 i zed. 


i 


i 

h 


» 


C33  Although  automatic  methods  which  precisely  define 
this  selection  criterium  have  been  developed 
CNAG783,  the  results  presented  below  are  based  on 
local  minima  which  were  manually  selected  by  the 
author. 
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In  order  to  overcome  these  difficulties*  Weszka 
CWES743  proposed  adding  gradient  information  into  the 
histogram  by  reducing  the  relative  weight  of  histogram 
entries  which  exhibit  high  gradient  magnitudes  It  was 
hypothesized  that  pixels  at  edges*  where  the  gradient 
magnitude  is  large*  have  feature  activity  which  is 
between  the  peaks  of  feature  activity  associated  with 
the  regions  bounded  by  the  edge.  It  was  hoped  that  the 
lower  weighting  of  these  pixels  would  sharpen  the  peaks 
and  valleys  of  the  histogram  This  method  is  referred 
to  as  the  grao 1 en t-we i q h ted  h i stoqram  method  ( M2 ) . 

Another  method  proposed  by  Weszka  CWES753. 
referred  to  as  the  h iqh-orad ient  h i stoqram  method  (M3>< 
is  based  on  a complementary  point  of  view.  It 
considers  only  pixels  of  high  gradient  magnitude  in  the 
computation  of  the  histogram  Presumably,  these  pixels 
are  more  critical  in  the  selection  of  the  threshold 
since  they  bound  an  edge  which  should  be  detected  by 
the  threshold  selected.  This  method  is  similar  to  the 
methods  of  Katz  CKAT65D  which  also  utilizes  only  the 
high  gradient  pixels.  The  Katz  method  computes  the 
threshold  simply  by  averaging  all  of  the  h i g h-grau i ent 


pixels  and  is  the  only  method  discussed  which  does  not 
utillize  a histogram.  The  Katz  method  ( M5  > is  referred 
to  as  the  h iqh-qrad ient  pixel  average  method 
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Another  method  for  selecting  a threshold 

utillizing  gradient  information  u»as  proposed  bg 
Watanabe  IWAT74D  This  method  selects  the  threshold  at 
that  intensity  value  for  which  the  total  gradient 
magnitude  is  largest  and  is  therefore  referred  to  as 
the  total  gradient  h 1 stoaram  method  <M4).  This  method 
and  the  modification  proposed  by  Weszka  CWES733 
represent  the  conceptual  starting  point  for  the 
threshold  selection  method  proposed  below  The 

Watanabe  method  estimates  the  expected  total  contrast 
of  all  edges  detected  by  each  threshold  and  the  Weszka 
method  normalizes  for  the  number  of  edges  detected 
This  is  the  only  method  discussed  so  far  which  selects 
a threshold  at  a peak  in  the  histogram  rather  than  at  a 
valley  Note  that  all  of  these  methodsi  except  the 
standard  histogram  method/  attempt  to  take  gradient 
information  into  consideration 


32  The  Proposed  Algorithm 

The  threshold  selection  method  proposed  in  this 
paper  attempts  to  incorporate  the  gradient  information 
much  more  directly  Unlike  the  previous  methods/  the 
algorithm  does  not  directly  depend  on  the  assumption 
that  regions  generate  peaks  in  the  histogram  of  feature 
values  Instead/  the  algorithm  is  based  on  the  obvious 
heuristic  that  edges  which  correspond  to  real  region 
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discontinuities  in  the  image  tend  to  have  high 
contrast,  while  edges  of  low  contrast  usually  do  not 
correspond  to  real  region  boundaries  This  is  the 
heuristic  on  which  most  of  the  edge  segmentation 
algorithms  are  based  Clearly,  this  heuristic  is  not 
valid  in  some  cases  — especially  when  high  contrast 
texture  is  present  — but  we  will  not  be  concerned  with 
these  cases  in  this  treatment  This  heuristic, 
translated  into  a threshold  selection  mechanism,  might 
be  stated  as  follows 

The  optimum  threshold  for  segmentation  of  the 
image  is  that  threshold  which  detects  more  high 
contrast  edges  and  fewer  low  contrast  edges  than 
any  other  threshold 

Under  the  assumption  above,  detecting  more  high 
contrast  edges  implies  a reduction  of  type  two  errors 
(missing  edges),  while  not  detecting  low  contrast  edges 
implies  a reduction  of  type  one  errors  (inserted 
edges  ). 

One  possible,  easily  computed,  function  which 
serves  as  a measure  of  the  criterion  stated  above  is 
the  average  contrast  of  all  edges  detected  by  a 
particular  threshold.  The  more  low  contrast  edges  a 
threshold  detects,  the  smaller  the  average  contrast 
becomes.  The  more  high  contrast  edges  a threshold 
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detects,  the  larger  the  average  contrast  becomes  If 

one  creates  a histogram  of  the  average  contrast  for 
each  possible  threshold  then  the  highest  peak  in  this 
histogram  corresponds  to  the  optimum  threshold 

The  use  of  the  average-contrast-histogram  was 
first  proposed  by  Weszka.  Vernon,  and  Rosenfeld  CWES733 
as  a possible  modification  of  the 

total-contrast-histogram  method  of  Watanabe  [WAT743 
Each  class  of  the  average-contrast-histogram  was 
defined  as  the  ratio  of  the  corresponding  classes  of 
the  tota 1-c ontrast  histogram  of  method  M4  and  the 
feature  value  frequency  histogram  of  method  Ml  This 
method  was  rejected  by  Weszka  (as  an  improvement  to  the 
Watanabe  method)  since  the  resulting  thresholds  were 
inconsistent  over  several  different  gradient  magnitude 
measures 

The  proposed  method  defines  the  average-contrast 
histogram  differently  from  Weszka  CWES733  as  follows 
Consider  an  edge  between  adjacent  pixels  a and  b where 
the  feature  value  at  a is  IlaD  and  the  feature  value  at 
b is  ICbl  and  (without  loss  of  generality)  ICal  s ICbl. 
We  define  the  edge  between  pixels  a and  b to  be 
detected  by  threshold  T if  and  only  if  I L a J v T *•  1 1 b 3 
We  can  then  define  the  number  of  edges  detected  by 
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threshold  T as 


1,  if  Itfl]  s T ■ Itbl 

N(T)  * ' f { a . b , t ) ! f < a , b , t ) =•  j 

all  1 0,  otherwise 

edges 

Similarly,  if  the  contrast  of  the  edge  between  pixels  a 
and  b is  given  by  c(a,b)  * ! Ital  - ILb3  I then  the 
total  contrast  detected  by  threshold  T can  be  defined 

as . 


Ic(a.b),  if  IIa.1  < T < ltbD 
C<T)  = ) g ( a . b . t ) ! g < a . b . t ) « j 

all  [0,  otherwi se 
edges 

The  average  contrast  of  all  edges  which  will  be 
detected  by  threshold  T would  then  be  C(T)  / N(T>  (or  0 
if  N(T)=0>  Note  that  each  edge  contributes  to  c(a.b) 
different  thresholds;  that  is.  as  the  contrast  c(a.b) 
increases,  the  set  of  possible  thresholds  which  detect 
the  edge  also  increases. 

Figure  3a  shows  how  a particular  edge  updates  C(T) 
across  the  set  of  possible  thresholds.  Since  for  each 
threshold  which  detects  the  edge,  the  contribution  to 
C(T)  is  c(a,b)  this  method  is  referred  to  as  the 
un i f orm  contrast  method  <M6).  This  method  implies  that 
any  of  the  thresholds  in  the  interval  (IlaJ.ICb])  are 
equally  acceptable  If  the  thresholding  step  was  the 
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last  step  in  the  segmentation  process,  then  there  would 
be  no  reason  to  differentiate  between  the  thresholds  in 
the  interval  (Ital.ICbl).  however,  the  thresholding 
process  is  followed  bg  an  error  correcting  relaxation 
process  discussed  in  sections  40  to  4.7  below  Let  us 
assume  that  the  edge  between  pixels  a and  b matches  the 
correct  segmentat i on;  that  is.  failure  to  detect  the 
edge  represents  a type  two  error  (Later  we  will 
define  a relaxation  process  which  operates  by  shifting 
the  feature  measurement  Itxl  to  increase  local 
consistency  ) Therefore  the  possible  thresholds  in 
(Ilal.  I C b ] ) are  not  equally  acceptable,  because  a shift 
of  ICal  toward  Itbl. implies  that  a threshold  near  ILa3 
would  result  in  a type  two  error  and  similarly  for  a 
shift  in  lib]  toward  ICal  Therefore,  the  threshold 
furthest  from  ILal  and  Itbl  should  be  favored  because 
it  will  tolerate  larger  changes  in  I L a 3 and  Ilb3  before 
it  f a i 1 s to  detect  the  edge  We  satisfy  this  criterion 
using  a definition  of  contrast  relative  to  the 
threshold 

c ( a.  b > =»■  M 1 N ( ! ICa]-T  ! , ! ICb3-T  ! > 

Figure  3b  shows  how  a particular  edge  would  update  C < 1 ) 
for  this  definition  of  contrast  Since  the  contrast  is 
defined  to  be  relative  to  the  threshold,  this  method  is 
referred  to  as  the  relative  contrast  method  (M7) 
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0.  3 Multiple  Threshold  Selection. 

The  proposed  threshold  selection  algorithm 
facilitates  the  selection  of  additional  thresholds 
For  any  initial  threshold  TC01.  all  edges  which 
provided  a non-zero  contribution  to  the  histogram  class 
corresponding  to  TCO]  are  detected  by  thresholding  the 


image  at 

TCO] 

We 

then  repeat  the 

calculation  of  the 

h i stogram 

as 

before 

except  that 

any 

edge 

a 1 r ead  y 

detec  ted 

by 

TCO] 

(or  in  g enera  1 . 

any 

previous 

threshold)  will  not  contribute  to  the  histogram.  Let 
us  assume  that  this  new  histogram  has  a peak  at 
threshold  Till  The  average  contast  of  edges  detected 
by  TCI]  can  be  no  greater  than  the  average  contrast  of 
edges  detected  by  any  previous  threshold  such  as  TCO] 
Clearly.  one  could  continue  to  select  thresholds  until 
the  maximum  average  contrast  for  any  threshold  fell 
below  some  threshold  8 > 1 This  8 is  referred  to  as 
the  minimum  average  contrast  criterion  for  threshold 
selection 

3 4 Threshold  Selection  Results. 

The  seven  different  algorithms  for  threshold 
selection  listed  in  Table  1 were  applied  to  three 
different  images  of  widely  varying  semantic  content  and 
complexity  The  simplest  image  shows  a white  blood 
cell  on  a dark  background  This  image  contains 
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relatively  little  texture.  The  white  and  red  blood 
cells  in  the  image  do  contain  internal  structure.  This 
image  is  typical  of  the  kind  of  images  for  which 
threshold  segmentation  methods  have  been  successfully 
utilized.  The  second  image  shows  a photo-micrograph  of 
a breast  duct  exhibiting  cribriform  morphology  (an 
abnormality  often  indicative  of  carcinoma).  This  image 
contains  much  more  texture  than  the  first  image  and  the 
foreground/background  relationship  is  much  less 
distinct.  The  third  image  is  a complex  outdoor  scene 
of  a house  with  bushes  and  trees.  This  image  contains 
a variety  of  objects  and  textures  with  no 
straightforward  f oreground/bac kground  distinction. 
These  three  images  are  shown  in  Figure  4. 

For  the  simple  blood  cell  images  all  of  the 
methods  except  methods  M4  and  M5  select  a threshold 
which  detects  the  boundary  of  the  nucleus  of  the  white 
blood  cell  (see  figure  5).  The  threshold  selected  by 
method  M5  detects  the  cytoplasm  boundary  of  the  white 
blood  cell  and  some  of  the  red  blood  cell  boundaries. 
The  threshold  selected  by  method  M4  is  too  low  and 
fails  to  detect  almost  all  of  the  important  edges  in 
the  image.  The  method  fails  since  the  histogram  is 
dominated  by  the  numerous  weak  texture  edges  found  in 
the  large  background  region.  The  histograms  for  method 


a)  white  blood  cell 


house  scene 


Three  Sample  Intake 
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Ml  and  M2  each  contain  two  distinct  valleys;  however, 
the  valleys  are  long  and  flat  making  selection  of  a 
threshold  difficult  The  multitude  of  local  maxima  and 
minima  make  threshold  selection  more  difficult  for 
method  M3  The  histograms  for  methods  M6  and  M7  are 
quite  smooth  and  selection  of  the  threshold  is  clear. 

Methods  Mi  and  M2  do  not  generate  good  thresholds 
for  the  breast  duct  image  (figure  6)  The  algorithms 
fail  since  the  assumption  that  image  regions  are 
uniform  and  therefore  generate  discrete  peaks  in  the 
histogram  is  violated  for  this  image  Note  that  the 
histograms  for  methods  Mi  and  M2  are  virtually  without 
any  valleys.  making  threshold  selection  almost 
arbitrary.  The  methods  M3.  M5»  M6.  and  M7  all  select 
thresholds  which  lead  to  reasonable  segmentations  All 
of  the  thresholded  images  exhibit  a plethora  of  texture 
edges  which  are  not  part  of  the  object  boundary 

All  of  the  algorithms  select  thresholds  which 
result  in  reasonable  segmentations  for  the  house  image 
(see  figure  7)  Each  method  detects  some  of  the  region 
boundaries  in  the  i ma  ge,  however,  none  of  the 
algorithms  detect  a high  percentage  of  the  semantically 
important  boundaries  Several  different  thresholds 
could  have  been  selected  from  the  histograms  generated 
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by  methods  Ml  and  M2.  For  method  MO  the  selection  of  a 
threshold  is  almost  arbitrary  due  to  the  numerous  local 
extrema  in  the  histogram  Selection  of  a threshold  for 
methods  M4,  M6  and  M7  is  not  difficult  since  the 
histograms  for  these  methods  exhibit  clear  peaks 


3.5  Multiple  Threshold  Selection  Results 

Figure  8 shows  the  multiple  threshold  selection 
process  for  the  three  images  using  method  M7.  The  top 
row  of  the  figure  corresponds  to  the  histograms  of 
figures  6.  and  7.  The  next  row  shows  the 
corresponding  histograms  generated  bg  method  M7  given 
that  ang  edge  detected  bg  the  threshold  selected  (based 
on  the  histogram  of  the  top  row)  is  ignored.  The  third 
row  shows  the  corresponding  histograms  computed 
ignoring  the  edges  detected  bg  ang  of  the  three 
thresholds  alreadg  selected  Up  to  five  thresholds  can 
be  selected  from  these  histograms  for  each  image  Only 
the  thresholds  which  meet  the  global  minimum  average 
contrast  constraint  are  utilized  Selection  of  this 
criterion  is  currentlg  ad  hoc  The  constraint  utilized 
in  the  following  experiment  required  an  average 
contrast  greater  than  2.0  The  bottom  row  shows  the 
resulting  region  labels  encoded  as  gray  levels  for  the 
three  images  thresholded  at  all  of  the  selected 
thresholds  These  images  preserve  most  of  the 
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important  boundaries  of  the  image  These  imaqes 
represent  a reduction  from  the  original  intensity  of  64 
distinct  labels  to  6,  4 and  6 distinct  labels  for  the 

blood  cell,  breast  duct,  or  house  image  respectively 

4.0  Threshold  Relaxation 

All  of  the  threshold  selection  algorithms 
discussed  in  the  previous  section  select  a threshold 
based  on  some  global  measure  Since  the  measure  is 
global  across  the  image,  it  is  probable  that  subareas 
exist  for  which  the  threshold  selected  is  not  optimal 
Furthermore,  the  measured  value  of  the  feature  is 
subject  to  errors  due  to  digitization  and  discrete 
repr esen ta t l on  of  the  image.  Together  these  factors 
imply  that  regardless  of  the  threshold  selection 
algorithm  utilized,  the  resulting  segmentation  often 
will  contain  many  of  the  three  error  types  previously 
discussed  The  type  two  errors  (missing  real  edges) 
are  reduced  by  the  selection  of  multiple  thresholds  as 
discussed  previously 

The  purpose  of  the  relaxation  procedure  is  to 
reduce  the  frequency  of  the  type  one  and  type  three 
errors.  The  relaxation  achieves  this  goal  by  utilizing 
the  local  context  of  each  pixel  to  determine  the 
relative  likelihood  that  the  pixel  is  correctly  or 
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incorrectly  labeled  For  the  purposes  of  this  paper, 
the  local  context  of  a pixel  is  the  neighborhood  of 
eight  1 mm  ediately  adjacent  pixels  The  process 
operates  by  updating  in  parallel  both  the  label  of  the 
pixel  and  the  feature  measurement  at  the  pixel  so  as  to 
reduce  local  inconsistency  The  process  terminates 
when  no  more  changes  are  possible 

The  relaxation  is  simply  an  interpolation  process 
between  the  feature  measurement  at  pixel  x and  the 
p a 1 1 a 1 context  of  x The  relaxation  is  based  on  the 
premise  that  pixel  x and  the  pixels  in  the  spatial 
context,  of  x probably  belong  to  the  same  region  in  the 
imaqe  and  therefore,  the  feature  mesurement  at  x and 
its  neighbors  should  be  on  the  same  side  of  the 
threshold  The  mechanism  for  reducing  local 

inconsistency  then  shifts  the  feature  measurement  at.  x 
toward  the  measurement  at  the  neighbors  in  order  to 
increase  the  likelihood  that  x and  its  neighbors  lie  on 
the  same  side  of  T This  interpolation  is  controlled 
based  on  the  following  additional  premises: 

The  likelihood  that  pixel  x and  pixel  y are 
part  of  the  same  region  is  inversely 
proportional  to  the  distance  between  x and  y 

3 The  likelihood  that  pixel  x and  pixel  y belong 
to  the  same  region  is  inversely  proportional 
to  the  likelihood  of  an  edge  existing 
somewhere  between  pixel  x and  pixel  y 

3 finally,  the  shift  of  the  feature  measurement 
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at  x should  be  inversely  proportional  to  the 
ambiguity  of  the  neighborhood 

The  third  premise  allows  the  relaxation  to  converge 

quickly  when  reliable  information  is  available  in  a 

subarea  of  the  image  These  areas  form  "islands  of 

r e 1 lab  1 i 1 1 ty " CLES77D  from  which  information  propagates 

to  the  adjoining  image  areas.  The  relaxation  does  not 

change  the  value  of  the  feature  measurement  until 

adequate  information  is  available  at  the  pixel. 

Table  2 below  summarizes  much  of  the  notation. 
The  neighborhood  of  pixel  x is  denoted  NCx.'l.  The  label 
of  the  pixel  LCxl  is  an  element  of  =<.0,  1>/  with 
LCxD=0  whenever  the  feature  measurement  ICxl  is  less 
than  or  equal  to  the  threshold  T.  and  LCxl  = 1 
o th  erwi se . 

The  neighborhood  elements  are  partitioned  into 
subsets  NCI#  xD  and  NCO#  x 3 . NCI#  x3  consists  of  those 
elements  of  the  neighborhood  labeled  1 while  NC0»xJ 
consists  of  those  elements  of  the  neighborhood  labeled 
0.  The  pixels  in  NCI#  xl  are  considered  to  be 
consistent  with  the  labeling  LC  x 3 = 1 and  inconsistent 
with  the  labeling  LCx3=0.  Likewise  the  elements  of 


NL‘0#x3  are  considered  to  support  the  labeling  LCx3=0 
and  to  contradict  the  labeling  LCx3  = l. 


Table  2 Threshold  Relaxation  Notation 


l The  set  of  labels  < 0,  1 > 


2 The  threshold  used  is  T. 


3.  The  feature  measurement  at  pixel  x is  I L x 3 


4 The  label  at  pixel  x is  LLx3>  where 
L C x ] = 0 if  I C x 3 s T and 
L C x ] = 1 if  11x3  > T. 


5.  The  neighborhood  about  pixel  x is  NCx3.  Not 
that  x £ NC  x 3 


6.  The  set  NCO»  x3  = < gig  t NCx3  and  LLg3  = 0 > 
The  set  NC  1 » x 3 = T g ! g e NL  x 3 and  LLg]  = 1 > 

7.  Imin  is  the  minimum  measurement  of  feature  I 
angwhere  in  the  image. 

Imax  is  the  maximum  measurement  of  feature  I 
anguihere  in  the  image. 
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4 1 Estimation  of  Contextual  Support 

It  is.  of  course,  necessary  to  quantify  the  degree 
to  which  NCI.  x3  or  NCO,  x3  supports  or  contradicts  LCx3 
given  the  feature  measurements  at  x and  for  each  y in 
NLx3  The  first  step  is  to  define  a measure  of  the 
likelihood  that  a pixel  in  the  neighborhood  is 
correctly  labeled  based  only  on  the  feature  measurement 
at  the  pixel  The  measure  used  in  this  paper  is  the 
relative  distance  of  the  feature  measurement  from  the 

I 

threshold  For  an  element  y e NLO.  x3  the  relative 
distance  is  DCO.  y3  = ( T-I C y 3 ) / ( T-Imin ) . For  an  element 
y < Nil.  x3  the  relative  distance  is  given  by 
DCl.yl  = ( I C y 3-T > / ( I ma x -T ) Note  that  the  relative 

distance  is  1 when  the  feature  measurement  is  as  far 
from  the  threshold  as  possible  and  0 when  the  feature 
measurement  is  at  T.  One  can  then  define  measures  of 
the  net  contextual  support  SCO.  x3  and  SCI.  x3  over  the 
neighborhood  as 

SCO,  x3  = I DCO.  y 3 WCx,y3  GCx,y3 
y € NCO,  x 3 

SCI,  x 3 = l DC  1 , y 3 WCx,y3  GCx,y3 
ye  NC  1 > x 3 

This  measure  represents  the  summation  of  support  for 
LCx3=0  and  LCx3=l.  respectively,  over  the  entire 
neighborhood.  The  WCx,y3  term  is  an  inverse  function 
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of  the  distance  between  pixels  x and  y The  specific 
function  WCx.yT  utilized  for  the  nine  pixel 
neighborhood  is  discussed  in  more  detail  in  section  4 5 
below  This  term  is  based  upon  the  assumption  that  x 
and  y are  less  likely  to  belong  to  the  same  region  as 
the  distance  between  x and  y increases.  The  GCx,y3 
function  is  used  to  consider  the  effect  of  boundaries 
on  the  net  contextual  support  measure  This  term 
accounts  for  the  assumption  that  the  larger  the 
intensity  or  feature  gradient  between  pixels  x and  y. 
the  more  likely  it  is  that  the  hypothesis  that  x and  y 
belong  to  different  regions  is  valid  The  gradient 
measure  GCx,y3  is  discussed  in  more  detail  in  section 
4 6 b e 1 ow 

42  Estimation  of  Relative  Confidences  for  LCx3. 

The  net  contextual  support  measures  SCOT  and  SC13 
can  easily  be  converted  to  the  relative  confidences  of 
labels  LCx3=l  and  LCx3=0  by  normalization. 

P (LC  x 3 = 1 ) = SCli  x I / ( SCI.  x3+SCO. x3  ) 

P <LC  x 3=0)  * SCO,  x3  / ( SC  1 . x 3+SCO.  x 3 ) 

These  confidences  represent  the  likelihood  that  x is 


correctly  labeled  LCx3=l  or  LCx3=0  based  on  the 
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information  in  the  neighborhood  of  * 


4 3 Proposing  an  Update  of  I C x 3 


If  x is  correctly  labeled  LCx3=i,  then  x should  be 
"close"  ( in  feature  value  ICx3)  to  all  of  its  neighbors 
which  are  also  labeled  l.  We  therefore  propose  a new 
value  I ' C •> 1 for  ICx3  based  on  the  gradient  and  distance 
weighted  average  of  the  consistently  labeled  neighbors. 

I Uy]  GCx.  y 3 WCx, y3 
y - NCO,  x 3 

AVECO,  x 1 * 

) GCx,  y 3 Wtx,  yl 
y < NCO,  x 3 

l ICyl  GCx,  y J WCx,  yl 
y < NC 1 , x J 

AVEC  1 . x 3=  

) GCx,yl  Wl  x , y 3 
y < NCI.  xl 

If  i*l  then  the  proposed  value  I ' C x 3=AVtC 1 . x 3 would 
necessarily  exceed  T and  x would  be  correctly  labeled 
Likewise  for  i=0,  the  proposed  value  of  I ' C x 3=AVEC0,  x 3 
would  be  less  than  T and  again  x would  be  correctly 
labeled.  Since  we  do  not  know  apriori  what  the 

"correct"  label  at  x is.  we  use  the  relative 

confidences  P(lCx3=l)  and  P(LCx3=0)  to  estimate  the 
confidence  of  a correct  label  of  1 and  0 at  x based  on 
information  in  the  neighborhood  of  x.  This  suggests  a 
value  of  I'Cx3  based  on  an  interpolation  between 


L 


i 


A SEGMENTATION  algorithm  based  ON  THRESHOLDING  PAGE  24 


I 

! 


AVEtO.  X I and  AVEC1.  xj  utilizing  the  relative 
confidences  as  follows 

I'txl  = P<Lr*]=l)  AVECl.xl  P (LC  x 3=0 ) AVECO,  xl 

Note  that  when  P<LCx3=i)  approaches  one.  the  value  of 
I'CxD  is  dominated  by  AVEC i . x 3 and  therefore.  I ' C x 3 
will  be  on  the  correct  side  of  T.  However.  if 
P<LCx3=l)  P(LCx3=0)  then  I'[x3  may  not  be  meaningful 
since  it  is  based  on  contradictory  evidence 


4 4 Using  Local  Ambiguity  to  Control  the  Update  of  ILx3. 

We  argue  that  the  magnitude  of  the  change  in  Itxl 
should  be  a function  of  the  ambiguity  in  the 
neighborhood  High  ambiguity  exists  when  the 
difference  between  P(ltx3=l>  and  P(LCx3=0)  is  small. 
When  the  ambiguity  is  large  there  is  not  enough 
information  in  the  neighborhood  to  be  sure  of  what  the 
best  final  labeling  of  the  pixel  should  be  The 
strategy  that  we  employed  will  shift  the  feature  value 
by  only  a small  amount  for  ambiguous  pixels  This 
effectively  suppresses  changes  at  a pixel  until  enough 
information  has  propagated  into  the  neighborhood  of  the 
pixel  to  make  the  decision  unambiguously  On  the  other 
hand,  when  the  relative  magnitude  of  either  label  is 
large.  the  relaxation  should  quickly  converge  to  the 
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correct  label  and  appropriate  feature  value 

The  ambiguity  measure  used  is  referred  to  as  the 
confidence  of  sufficient  inf ormation  and  is  given  by 

CSIExl  = ! P(LEx3=l>  - P < LE  x 3=0  > ! . 

The  compliment  of  this  measure/  the  degree  of  ambiguity 
or  the  confidence  of  insuf f ic  ient  in  format i on  is  given 
by 

CIIEx3  = 1.0  - CSI  C x 3 

The  feature  update  function  can  now  be  expressed  as 
foil ows : 

I E x 3 <=  CIIEx3  I E x 3 + CSI  E x 3 I'Ex3. 

The  ambiguity  factors  CIICx3  and  CSIEx3  balance  the 
effect  of  the  current  and  proposed  values  of  IEx3. 
Note  that  by  the  definition  of  the  ambiguity  terms  the 
change  in  IEx3  can  be  large  only  when  either  P(LEx3=l) 
or  P(LEx3=0)  predominates.  The  net  effect  of  the 
ambiguity  process  is  to  shift  the  feature  value  slowly 
when  there  is  uncertainty  and  more  rapidly  as  either 
label  begins  to  clearly  dominate. 


4.5  The  Distance  Weighting  Function  WEx>y3. 

The  function  WEx.y3  represents  a spatial  weighting 


over  the  neighborhood  about  x based  on  the  distance 


r ^ 
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between  pixel  x and  pixel  y in  NCx3.  The  value  of 
WCx.yD  is  defined  to  be  proportional  to  the  inverse 
distance  squared.  Typically,  the  metric  used  for  the 
distance  is  the  Euclidian  distance  between  the  pixel 
centers.  Unfortunately,  using  this  metric  results  in 
infinite  weight  for  the  central  pixel  x since  the 
distance  is  0.  This  implies  that  the  neighboring 
pixels  should  have  zero  weight  after  normalization  and 
contradicts  the  assumption  that  the  neighborhood  of  x 
contains  useful  information. 


The  digitization  process  does  not  represent  a 
point  measure  at  the  center  of  a pixel,  but  an  average 
measure  of  the  feature  integrated  across  the  spatial 
context  of  the  pixel.  The  proposed  distance  metric 
will  be  based  upon  the  average  distance  of  the  spatial 
area  from  the  center  point  of  the  center  pixel  (see 
figure  9).  This  measure  utilizes  the  measure  of 
distance  across  the  entire  area  for  which  the  feature 
value  was  computed.  The  simple  integration  assumes 
that  the  feature  measurement  is  based  on  a uniformly 
weighted  average  across  the  pixel.  With  additional 
information  about  the  sampling  characteristics  of  the 
digitization  device,  more  precise  coefficients  WEx.y] 


could  be  computed. 


1)  The  average  distance  from  the  origin  for  the  center  pixel  is 

1 


8 ' 


0 = 0, 


2 cos  0 


r dr  ilO  = 1 


r » 0 


2)  The  average  distance  tor  the  side  adjacent  pixel  is 


r 0 “ 0 1 


1 


2 cos  0 


j 

0 = 0 r 


r dr  d0  + 2 


I 


0 = 0, 


2 cos  0 


1 

sin  9 


0 = 0,  r 


> 1 


r dr  dO  = 3/3 


2 cos  0 


3)  The  average  distance  for  the  corner  adjacent  pixel  is 
2 '°  “ °2  Jr  “Tcosl 

0 ’ 0 1 r " 2~s  i7fo 


r dr  dO  = 3/2 
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For  the  neighborhood  consisting  of  the  pixel  and 
its  eight  adjacent  neighbors,  there  are  three  cases  to 
be  consisdered: 

I)  the  central  pixel,  2)  the  edge  adjacent  pixels,  and 

3)the  corner  adjacent  pixels.  Assuming  Wtx.yJ  varies 
inversely  with  the  square  of  the  distance,  the 
normalized  values  for  the  weights  Wtx.yl  corresponding 
to  these  three  cases  are  simply:  1 for  the  central 
pixel,  9/25  for  the  edge  adjacent  pixels,  and  4/25  for 
the  corner  adjacent  pixels. 


4 6 The  Gradient  Weighting  Function  GLx.yJ. 

The  degree  to  which  pixel  y in  NCx3  should  be 
utilized  to  update  x is  a function  of  the  likelihood 
that  x and  y are  part  of  the  same  object  It  will  be 
assumed  that  the  larger  the  magnitude  of  the  local 
gradient,  gCx.yl,  between  pixels  x and  y,  the  less 
likely  it  is  that  x and  y are  part  of  the  same  region 
in  the  image,  which  implies  that  GIx.yD  should  decrease 
as  gCx.y]  increases.  For  the  purposes  of  this  paper, 
glx.yl  is  defined  as  simply  the  feature  difference  of 
the  pixels:  gCxy]=  ! ICxD— ICg3  ! . It  is  also 
hypothesized  that  for  some  K if  gtx.yl  > K.  the 
influence  of  y on  x is  negligible.  The  value  of  K was 
selected  such  that  957.  of  all  gradients  in  the  image 
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were  smaller. 


A third 

assump  1 1 on 

which 

1 s 

embedded  in  the 

definition 

of 

GCx.yD  states 

that 

any 

region  which 

consists  of 

only  a single 

pixel 

1 s 

of 

no  interest. 

This  requires 

that  there 

exists 

some 

y in 

NE  x 1 , y =*  x > 

such  that  * and  y are  members  of  the  same  region. 

Now  let  us  define  G(x,y)  in  terms  of  g<x.y).  The 
heuristics  can  be  translated  into  the  following 
gradient  weighting  function 


r 1, 

i f 

x=y 

GE  x.  y 1 = ■ 

1, 

if 

gmax=gmin 

gmax  - 
[gmax  - 

g E x > y 1 i 

gmin 

otherwise 

where  gCx»yI=  ! IExl  - IEyl  ! » 

gmax  - Max  ( K » gEx.yl)  . and  gmin  = Min  ( gtx.y]) 
y c NC  x 1 y . NLxI 

This  function  sets  GEx* yJ=l  for  y = x.  This 
formulation  also  guarantees  that  for  at  least  one  of 
the  neighbors  of  x<  GEx>y]  will  be  1.  in  particular 
that  neighbor  y in  NCxl-x  which  is  most  like  x.  If  all 
points  in  NExl-x  are  equally  like  x.  then  all  will  have 
GEx»ylsl.  The  function  is  zero  only  when  gEx<yl  is 


equal  to  the  largest  local  gradient  and  gtx>yl  is  at 
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least  as  large  as  K 


4.  7 Threshold  Relaxation  Results. 

Figure  10  shows  the  operation  of  the  threshold 
relaxation  process  on  one  of  the  binary  labelings  of 
the  house  scene  <T=29>.  The  number  of  regions  present 
is  greatly  reduced  by  the  process  Isolated* 
unsupported  pixels  quickly  shift  accross  the  threshold, 
in  fact  many  of  these  pixels  are  'corrected'  during  the 
first  iteration.  As  expected,  larger  regions  which  are 
strongly  supported  by  the  image  data  are  changed  very 
little  by  the  process  (note  the  leftmost  bush  in  figure 
10).  However,  when  the  exact  position  of  the  boundary 
for  a region  is  not  clearly  determined  by  the 
threshold.  the  boundaries  of  the  region  may  shift 
considerably  (note  the  window  above  the  leftmost  bush). 
These  changes  take  place  much  more  slowly  since  the 
boundary  pixels  must  wait  for  information  to  propagate 
before  the  decision  at  the  pixel  is  unambiguous.  Even 
after  30  iterations  the  process  had  not  fully 
converged. 

Figure  11  shows  the  three  segmented  images  before 
and  after  the  relaxation  process  Each  gray  level 
encodes  a label  of  the  set  of  labels  produced  by 
intersection  of  the  binary  thresholded  images  In  each 


After  Iteration  30 


After  iteration  5 


Figure  10. 
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caie  the  relaxation  reduces  the  number  of  distinct 
regions  in  the  segmentation.  Mang  of  the  small  regions 
eliminated  were  undoubtablg  of  no  semantic  interest  and 
thus  represent  the  elimination  of  tgpe  one  errors  in 
the  segmentation.  Shifting  the  edges  slightlg  to 
enhance  local  consistency  should  also  have  reduced  the 
frequency  of  type  three  errors  The  figure  clearly 
shows  that  the  relaxation  process  improved  the  quality 
of  the  segmentations 

Note  that  the  total  amount  of  change  induced  by 
the  relaxation  process  varies  with  the  complexity  and 
level  of  texture  of  the  images  For  the  simple  image 
of  the  blood  cell  the  initial  thresholds  provide  a 
segmentation  which  bears  little  improvement  by  the 
relaxation  process.  On  the  other  hand,  for  both  the 
breast  duct  and  house  images  the  initial  threshold 
labeling  makes  many  apparent  errors  especially  in  areas 
of  high  texture.  Many  of  these  errors  are  corrected  by 
the  relaxation. 

5.  0 Summary. 

Based  on  the  authors  subjective  evaluation.  the 
proposed  segmentation  algorithm  resulted  in  acceptable 
segmentations  for  the  three  test  images  of  varying 
complexity  The  first  phase  of  the  segmentation 
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algorithm  generated  an  initial  aagmantation  by 

selecting  a *at  of  thresholds  based  on  local  adga 
information  in  tha  imaga.  Tha  proposed  threshold 
selection  method  compared  favorably  with  alternative 
threshold  selection  methods  The  initial  segmentations 
contained  many  type  one  and  type  three  errors  The 

selection  of  multiple  thresholds  helped  to  reduce  the 
frequency  of  type  two  errors.  The  relaxation  process 
improved  the  quality  of  the  final  segmentation, 

reducing  the  complexity  of  the  segmentation  by 
eliminating  many  of  the  type  one  and  type  three  errors 
The  re.  ixation  process  utilized  both  local  edge 
information  and  pixel  similarity  information  to  modify 
the  segmentation. 


Although  the  results  have  been  encouraging,  it  is 
not  clear  to  what  range  of  images  this  approach  is 
applicable.  One  weakness  of  the  current  implementation 
is  that  only  a single  image  feature  is  used  to  generate 
the  segmentation.  Future  work  will  consider  the 
automatic  threshold  selection  generalized  over  a pool 
of  features  where  the  image  feature  and  threshold  are 
selected  together  for  each  threshold.  This  will  allow 
for  utilization  of  color  information  now  ignored  by  the 


process 
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"The  segmentation  algorithm  proposed  in  this  paper  is  a complex  form  of 
thresholding  which  utilizes  multiple  thresholds.  The  algorithm  consists  of 
two  major  components:  a threshold  selection  component  and  a relaxation 
component . 


The  threshold  selection  component  automatically  selects  a threshold  so 
as  to  maximize  the  global  average  contrast  of  edges  detected  by  the  threshold 
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across  the  Linage.  This  algorithm  for  threshold  selection  compares  favorably 
with  other  methods  for  automatic  threshold  selection.  The  threshold  selection 
algo i ' thm  can  be  applied  recursively  to  select  additional  thresholds  by 
ignoring  any  edges  which  have  already  been  detected  by  previously  selected 

thresholds. 

The  relaxation  component  utili-es  the  immediate  spatial  context  of  each 
pixel  to  update  both  the  label  at  tae  pixel  and  the  feature  measurement  at 
the  pixel.  The  update  function  proposes  a new  feature  value  at  tlu>  pixel 
defined  by  a weighted  average  of  the  central  pixel  and  all  of  its  neighbors. 

The  weight  associated  with  each  pixel  (with  respect  to  the  pixel  being  updated) 
is  proportional  to  the  spatial  distance  between  the  pixels,  the  probability 
that  the  two  pixels  are  correctly  labeled,  and  the  probability  that  the  two 
pixels  belong  to  the  same  region.  The  update  function  then  replaces  the 
feature  value  at  the  pixel  with  a value  somewhere  between  the  current  value  and 
the  proposed  value.  When  the  local  evidence  for  shifting  the  feature  value  is 
consistent,  then  the  value  adopted  will  be  close  to  the  proposed  value; 
however,  when  the  local  evidence  is  inconsistent,  the  value  adopted  will  be 
close  to  the  original  value. 

^ The  relaxation  is  independently  performed  for  each  threshold  selected. 

The  resulting  binary  images  are  intersected  to  produce  the  final  segmentation. 
This  algorithm  works  well  not  only  for  simple  images  but  also  produces 
reasonable  segmentations  for  complex  images. 
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